Skip to content

backlog: DRAFT-14 — per-group transformer fitting - #13

Closed
ahrzb wants to merge 1 commit into
masterfrom
pm-board-draft-14-per-group-fitting
Closed

backlog: DRAFT-14 — per-group transformer fitting#13
ahrzb wants to merge 1 commit into
masterfrom
pm-board-draft-14-per-group-fitting

Conversation

@ahrzb

@ahrzb ahrzb commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Board-only change (backlog/**). One new draft, no code.

DRAFT-14 — transformer(x) OVER (PARTITION BY g) fits one transformer per group

AmirHossein's proposal, routed via Wren out of TASK-3 brainstorming. Wren's measurements are what make it scopeable, so they're recorded verbatim in the ticket.

The reframe that matters: today a transformer ref is fitted once globally — measured, with country a={10,20}, b={30,50} the state holds avg_age=[27.5], not per-country 15/40. That is the correct semantic (it matches sklearn, where a Pipeline step fits once on all training data), so this is a new feature, not a bug.

The split, which is the spine of the ticket:

  • (a) SQL-expressible (StandardScaler, MinMax, MaxAbs, mean-SimpleImputer) — already works end-to-end today. Pure desugar, zero engine work. Measured: the PARTITION BY form yields one state row per group, batch == infer, and unseen groups return NULL for free via the existing unseen-partition semantic.
  • (b) Opaque sklearn objects — genuinely new. N fitted clones keyed by group, per-group state through the artifact and lookup path.

Four open questions, why it's a draft not a task:

  1. Unseen-group policy for (b) — NULL, global fallback, or error? doc-2 calls unknown-category handling "a designed-in requirement, not a flag." Wren's point: (a) inherits NULL by construction, so any other choice for (b) creates a semantic divergence between two halves of one feature.
  2. Syntax — OVER (g) shorthand vs standard OVER (PARTITION BY g). Wren leans standard; (a) then desugars into already-valid SQL, whereas a shorthand buys new parsing for no new capability.
  3. Ship (a) alone first? Nearly free, covers the common case.
  4. Artifact size for (b) — N clones grows the artifact with group cardinality.

Explicitly flagged as distinct from DRAFT-11: that one is about how arguments bind; this is about what data a transformer is fitted on.

🤖 Generated with Claude Code

@ahrzb

ahrzb commented Jul 23, 2026

Copy link
Copy Markdown
Owner Author

Superseded by #14, the rolling board PR — same DRAFT-14 commit, now on the standing pm-board branch so board changes accumulate in one place instead of a PR per change.

@ahrzb ahrzb closed this Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant